1. Field of the Invention
The present invention generally relates to geospatial data collection with cameras and other devices and, more particularly, to a method and corresponding system for solving for an initial azimuth to assist in processing data sets (e.g., position information and images from one or more cameras) to create geopositional databases allowing accurate measurement of points and objects in three dimensional (3D) space (e.g., determination of 3D locations of objects in a panoramic image captured with a set of panoramic cameras that may be provided as part of a survey instrument or other electronic device).
2. Relevant Background
Traditional surveying may involve operators working with a theodolite and range pole. One operator generally positions the theodolite over a known point while the other operator holds the range pole at a series of points whose locations are to be determined. A target mounted on the range pole is sighted through the theodolite and accurate angular measurements are obtained. Accurate distance measurements may also be obtained using a measuring tape or any other distance measuring device. The locations of the points can be determined using the angular and distance measurements in accordance with known triangulation techniques. Modern surveyors typically use an instrument called a total station, which can be thought of as a theodolite with a built-in laser range finder. Such an instrument is more convenient because distances can be measured directly instead of requiring triangulation techniques or a measuring tape. Surveying with total stations is usually most optimal when the range pole is equipped with a prism to reflect the light from the laser range finder back to the total station. This helps to avoid accidentally measuring to unintentional objects.
In modern satellite-based surveying, an operator may move about with a survey instrument that determines position information using a global navigation satellite system (GNSS). This is typically a GNSS receiver, which may be referred to as the rover. The satellite positioning system most commonly used today is the global positioning system (GPS), although others such as the global orbiting navigation system (GLONASS) are in use or under development. The operator generally moves about a site and stops at various points to record location information determined using signals transmitted by satellite sources to a receiver on the portable survey instrument. Location information is automatically calculated using a differential technique that requires a second GNSS receiver known as the base station to be located at a known position. It collects satellite signals at the same time as the rover. Satellite signals common to both receivers are used to compute a difference or delta vector. When this delta vector is added to the known position of the base station, the rover position is known to a much higher accuracy than a single receiver could accomplish. This can be done in real-time (real-time kinematic (RTK)) or in a post-processing step if more accuracy is needed.
Both traditional and satellite-based surveying are methods of determining location information for points that are typically on the ground or stationary objects. However, for smaller areas, a simple tape measure can also be used. It does not matter for the remainder of this document how these locations are measured.
In photogrammetry-based surveying, an operator may move about a site with a survey instrument that includes one or more cameras. Hereinafter, the term “survey instrument” shall generally refer to this type of instrument. For location information, it may be coupled with a GNSS rover receiver or a prism for a total station to measure. It may have neither, though, and rely on other techniques such as a tape measure, to supply location information. Such an instrument is very similar to a theodolite because it is only capable of measuring angles. One advantage of photogrammetry-based surveying is that it is capable of capturing large amounts of data in a short time, but a disadvantage is that it requires more effort to process the captured data.
The survey instrument may also include tilt sensors that provide information on rotation of the survey instrument (i.e., X and Y rotation) at the time the photographs are gathered by the operator. A compass can also be included in the survey instrument to provide additional instrument orientation information. Multiple cameras are often used with modern survey instruments to increase the amount of image information or photographs captured for a surveyed site. For example, some survey instruments includes a set of cameras (e.g., 7 to 12 cameras or the like) that are used to capture 360-degree digital panoramic images for efficient visual documentation and measurement of the surrounding environment near the survey instrument (e.g., to capture a continuous view about the survey instrument).
The combined information can be used with known photogrammetry techniques to determine 3D coordinates of points. Such techniques often use two or more overlapping or adjacent panoramic images taken from different stations (e.g., different locations). The location and image information can be processed at the site using a controller or may be processed at an office remote from the site using a computer system (e.g., a central server processing the data to create a geodatabase or database populated with geospatial documentation). The image information in the geodatabase can be exported for additional analysis (e.g., to modeling software) and allows for additional measurements to be obtained at any time as long as the additional points or objects are visible in the images.
For a surveyor, the amount of time spent on the job site may mean the difference between profit and loss. With modern survey equipment using photogrammetry techniques, there is little need to return to the field or the site to collect additional data. Instead, the site is completely documented with the 360-degree panoramic images as well as its spatial relationship so that any “missed data” can be quickly and efficiently created from the geodatabase created from data gathered by the survey instrument with its cameras and other sensors (e.g., a position receiver, tilt sensors, and a compass). With the ability to simultaneously capture images and sensor information for the survey instrument, objects at the site or surveyed environment can be quickly located with software at a central site (e.g., at the home office via a network connection to the server running the photogrammetry software). So, the surveyor can not only visualize the surveyed site, but they can also measure it (e.g., selection of a point on an image can result in position information being provided). Stated differently, a point cloud can be generated from the set of collected images along with their position data, and, then, a 3D model can be built from the point cloud.
Survey instruments are just one example of a camera or camera-based device that is equipped with orientation sensors, and when combined with the use of an real time kinematic (RTK) or similar receiver, the position as well as the orientation of each photograph captured by the camera (or each camera of a multi-camera survey instrument) is known or can be determined to a moderate degree of accuracy. With each of these devices, a bundle adjustment is always used as part of the processing of the collected data set to improve the accuracy so, for example, that close-range surveying can be done from the photographs/captured images. The bundle adjustment calculates the pose (position and orientation) for the camera for each image and must have tie-points to work properly or to work at all. A tie-point is a 3D location that is calculated from two or more observations. A tie-point observation is a row and column coordinate on an image that corresponds to a stationary object seen in the image. It also includes a tie-point identification (ID) and image identification (ID).
The bundle adjustment can prove problematic for a number of reasons. As the number of cameras in a device or cluster of images increases, it can become increasingly difficult to find the tie-points. Automatic tie-point finding algorithms have been developed and implemented, and automatic tie-point finding algorithms may deal with this problem by requiring moderately accurate initial positions and orientations. In many camera-based devices, the azimuth component of orientation is measured by an onboard compass or compass sensor. Practically speaking, there is not a better choice than a compass.
Unfortunately, the use of the compass sensor to find the azimuth can cause issues because a typical compass sensor is easily affected by even very small magnetic anomalies resulting in an inaccurate or completely incorrect azimuth value. When instrument locations are measured with a tape measure, the orientation of the coordinates can easily conflict with the compass sensor thus giving the result that the compass appears to be completely incorrect. Then, during the bundle adjustment, the automatic tie-point finder, using these erroneous azimuth values, will fail an unacceptable percentage of the time according to many users of this software. Additionally, the bundle adjustment algorithm can take a long time to run only to determine that the compass error (or azimuth error) is too large for a successful result. In some cases, manual tie-points in an image(s) can be entered or selected by a user/viewer when the automatic tie-point finder fails, but this can be a time consuming process that may need five to eight or more tie-points to detect blunders, and it may take a user an unacceptably long time to enter these manual tie-points, e.g., 20 to 40 minutes in many cases.
Hence, there remains a need for an improved method for determining initial azimuth when processing geospatial data from a camera (e.g., from cameras of a survey instrument) or other electronic device. Preferably, this method would be adapted to overcome the issues that can cause existing bundle adjustment methods (or software modules/routines) to fail by addressing the unreliability of the azimuth information such as that provided by a compass sensor. Generally, position and tilt information are much more reliable than azimuth information so the need is mainly to solve the azimuth. Do so will allow the bundle adjustment to be more efficiently performed as all six of its input values will be reliable, which will greatly improve determinations of camera poses (e.g., calculated positions of the camera are used to override values from the camera or other device's sensors) and will facilitate calculation of a point cloud from images and associated position/orientation data.